Survey of Experience Replay

Survey of Experience Replay

Survey of Experience Replay

Survey of Experience Replay Hiroyuki Yamada List of Papers Surveyed H. Liu et. al., “Competitive Experience Replay”, ICLR (2019) ( arXiv) G. Novati and P. Koumoutsakos, “Remember and Forget for Experience Replay”, ICML (2019) ( arXiv, code) D.

Speed up Ape-X implementation on single machine

Alghough neural network is optimized for GPU, environments for reinforcement learning (e.g. simulator) are not always GPU friendly. One of the method to speed up reinforcement learning is to run

DeepMind's reinforcement learning framework: Acme

As I mentioned at the previous post, DeepMind published a reinforcement learning framework Acme. 1. Introduction Acme provides higher level API, a simple training code can be something like following;

[gnwrapper] Support KeyboardInterrupt on Google Colab

Gym-Notebook-Wrapper (aka. gnwrapper) is a Python package to render OpenAI Gym on Google Colaboratory (or Jupyter Notebook at Linux). A class gnwrapper.Monitor wrapps gym.wrappers.Monitor and adds usuful features (e.g. starting

DeepMind/Reverb usage

Following the previous post, I write about DeepMind/Reverb, an experience replay framework. When reading source code, I found Reverb has other ways of adding and sampling transition data. Reverb has

DeepMind released new library for Experience Replay: Reverb

On 26th May, DeepMind released a new library, Reverb, for experience replay. Reverb is an efficient and easy-to-use data storage and transport system designed for machine learning research. Reverb is

Run and Render OpenAI Gym on Google Colab (Gym-Notebook-Wrapper)

Intro Google Colab is very convenient, we can use GPU or TPU for free. However, since Colab doesn’t have display except Notebook, when we train reinforcement learning model with OpenAI

Gym Notebook Wrapper

Wrapper for running and rendering OpenAI Gym on Jupyter Notebook

Google Releases Scalable RL architecture SEED RL (with yet another Replay Buffer implementation)

On 23rd March 2020, Google released new scalable distributed reinforcement learning architecture “SEED RL (Scalable, Efficient Deep-RL)” ( official blog, paper). In this archtecture, a model (aka. deep neural net)